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Abstract 

We present some results concerning the almost sure behaviour of the 
operator norm or random Toeplitz matrices, including the law of large 
numbers for the norm, normalized by its expectation (in the i.i.d. case). 
As tools we present some concentration inequalities for suprema of empir- 
ical processes, which are refinements of recent results by Einmahl and Li. 
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1 Introduction 

For a sequence of independent real random variables Xq,Xi, 
associated sequence of random Toeplitz matrices, given by 
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In two recent articles, Bryc, Dembo and Jiang [5] and independently Ham- 
mond and Miller [9] addressed the questions raised previously in Bai's paper 
[3], concerning the asymptotic behaviour of the spectral measure of random 
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Toeplitz matrices generated by i.i.d. mean zero, variance one random variables. 
They showed that the spectral measure of n~ x l 2 T n with probability one con- 
verges weakly to a nonrandom measure, independent of the distribution of the 
underlying i.i.d. sequence. Their results were later complemented by Bose and 
Sen g] and Meckes [T3] who analyzed the asymptotic behaviour of the spectral 
norm of a random Toeplitz matrix. Bose and Sen proved a law of large numbers 
in the case of i.i.d. random variables with positive mean and finite variance. 
Meckes showed that if all X^s are centered and uniformly subgaussian then 
E||T„|| ~ y/n \ogn and the same holds with probability one for ||T„||, provided 
that Xi's have some concentration of measure property. 



In the present article we are interested in extending the results of [T2] to 
i.i.d. mean zero random variables with finite variance as well as to independent 
non-identically distributed variables, which satisfy some moment bounds. In 
particular, we prove that if X$, Xi, X%, . . . are i.i.d. random variables, EXq = 
and < EXq < oo, then 

Pn|| 1 

hm = 1 a.s. 

n^oo E||T„| 

This is the statement of Theorem [5] in Section 13.2.21 

The above result, together with some additional arguments, may be used to 
prove that if Xq, X\, X%, . . . are i.i.d. random variables, then 

limsup i== < oo a.s., (1) 
n->oo Vnlogn 

if and only if EX = and EXq < oo, which is the content of Corollary [31 
Section [3X1 



In the framework of independent, but not necessarily identically distributed 
variables, we give sufficient conditions for (JTJ to hold (Proposition [U Section 
l3~2Tl) 



Our tools come from the classical probability in Banach spaces theory and are 
based on concentration inequalities for sums of independent Banach space valued 
variables. In particular, the proofs are inspired by a recent paper by Einmahl 
and Li [B], in which a very general law of the iterated logarithm for sums of i.i.d. 
variables in Banach spaces has been characterized. The other crucial ingredient 
is the original argument by Meckes which is applied conditionally. 



The organization of the paper is as follows. In the next section we present the 
basic concentration results for sums of independent random variables. Next, in 
Section [3j we pass to the original problem, concerning random matrices. First 
we obtain some useful estimates for the expected operator norm and then we 
prove results on the almost sure behaviour. We conclude with two exponential 
inequalities. 



2 



In the article we will deal both with absolute constants and constants de- 
pending on some parameters, which will be explicitly stated or clear from the 
context. In both cases the values of constants may change from line to line. 
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2 Concentration of measure for suprema of em- 
pirical processes 

In this section we introduce the basic tool required for the analysis of the almost 
sure behaviour of ||T n ||, i.e. concentration inequalities for sums of independent 
Banach space valued random variables. The following exposition is motivated 
by the work of Einmahl and Li. In particular, our Theorem [2] is a version of 
Theorem 3.1. from [5J. 

All the concentration results presented below are based on the celebrated 
inequality by Talagrand 13]. We will use a recent version by Klein and Rio, 
which gives the optimal constant 2 in front of the variance. We would also like 
to mention that since the natural normalization of the operator norm of Toeplitz 
matrices is ^/nlogn, which is much bigger than the classical law of the iterated 
logarithm normalization, this optimal value of the constant is not important 
for our applications (in fact any other constant would work). However, since 
inequalities for sums of Banach space valued variables with optimal value of 
this constant are of independent interest and obtaining them does not require 
much additional work, we present the results of this section in a slightly larger 
generality, than necessary for subsequent applications. 



Let us first recall the result due to Klein and Rio 

Theorem 1 ([H], Theorems 1.1, 1.2). Let X\, X2, ■ ■ ■ , X n be independent ran- 
dom variables with values in a measurable space (S 1 , S) and let T be a countable 
class of measurable functions f : S — > [— M, M], such that for all i, E/(Xj) = 0. 
Consider the random variable 



Z = sup^/(X i ). 

* i=l 



Then, for all t > 0, 

F(Z >EZ + t) < exp 

and 

¥(Z < EZ - t) < exp 



2(cr 2 + 2MEZ) + 3Mt 



2(cr 2 + 2MEZ) + 3 Aft 
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wher 



The following corollary was derived from the above theorem by Einmahl and 
Li [5] (they stated only the first part, concerning the upper tail but their proof 
yields also the other inequality). 

Corollary 1. Let X±, . . . , X n be independent mean zero random variables with 
values in a separable Banach space B. Assume that \\XiWoo < M for every i 
and let 

a 2 = sup VE/(I,) 2 . 

/6BM|/||<1 i 

Consider the random variable 

n 

z = \\j2 x i\\- 

i=l 

Then for all < r] < 1, S > there exists a constant K — K{rj,S), such that 
for all t > 0, 

k ,9 

P( max || y^Xi\\ > (l+rj)EZ + t) <exp 

Kk<n ^- — ' 



2(l + 5)a 2 



exp 



t 



and 



F(Z < (l-rj)EZ-t) < exp 



exp 



K max,- 1 1 Xj 



t 



K max,- 1 1 Xi 



2(1 + 5)ct 2 J 

Using the above result, Einmahl and Li [5] obtained an infinite dimensional 
version of the Fuk-Nagaev inequality for the deviation of maxima of partial sums 
of independent random variables [7]- For our purposes we will need a slightly 
refined version of their estimate, given in the following 

Theorem 2. In the setting of Corollary^ assume that for some p > 1 and 
all i, E\\Xi\\ p < oo. Then for all < rj < 1, 5 > there exists a constant 
C = C{p, n, 5), such that for all t > 0, 

k 

P (^ X llE^H >{l+ri)EZ + t] 

\ Kk<n * — ' / 



< exp 



[ t_ 

L 2(1 + S)a 



CE max \\Xi\\ p /t p 

l<i<n 



(2) 



and 



»(z < (1 - r\)EZ - t 
< exp 



2(1 + 5)cr 2 



CE max ||X,|| p /t p . 

l<2<n 



(3) 



4 



The proof we give below follows quite closely the original argument of Ein- 
mahl and Li and is based on the Hoffmann- J0rgensen inequality. The difference 
is that we use it in the moment version, which in our opinion simplifies the 
argument and also allows to replace J2i '^■\\Xi\\ p , appearing in the original es- 
timate by Einmahl and Li, by Emaxj ||- 5 Q|| P (this will be useful when dealing 
with Toeplitz matrices generated by non i.i.d. variables). A somewhat simi- 
lar application to moment estimates appeared in [8], whereas [2] contains an 
analogous derivation in the context of exponential inequalities for unbounded 
empirical processes (its final version was also influenced by results of Einmahl 
and Li). We state it at the end of the present section. 

Proof. For e = e(6) > (whose value will be determined later) consider trun- 
cated variables Yj = Xil^ Xi \\< P }- Set also Ri = Xi — Yi. 
We have 

max || < ma? II " EK <)H + ™ x H TP* ~ ER ^ 

l<k<n £ — ' lS^__ n — l<k<n A — ' 

i—1 i—1 i—1 

and 

II X>ll> || - EYi)\\ - || - ERi)l 

i—1 i—1 i—1 

where we used the fact that EYi + EB4 = 0. 

Similarly, using Jensen's inequality, we obtain 



eh Y,( Y i - E ^)ii - 2E ii X>>n ^ EZ ^ E n J2( Y > - Ey *)n + 2E n S^ii- 

i—1 i—1 i—1 i—1 

Thus, setting A = Emaxi< fe <„ |j £\ =1 we get 

k 

P( max W^XiW >(l+ V )EZ + t) 

- - i=l 

k 

<P( max || V (Yi - EYi)\\ > (1 + v)EZ + (1 - e)t) 

Kk<n — ' 
_ _ i=l 

k 

+ P( max || V(i?i - ERi)\\ > at) 

- - i=l 

k k 

<P( max || - Ey )ll > (1 + VM _ E ^)H - 4^4 + (1 - e)t) 

~ ~ n i=l i=l 
k 

+ P( max || V(i? t - ERi)\\ > et) (4) 

Kk<n — 4 
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and similarly 

F(Z < (l-rf)EZ-t) 

n 

<P(|| ^(F, - EFi)|| < (1 - ri)EZ - (1 - e)t) 



+ P(||^( J R i -E J R i )|| >et) 
i=l 

n n 

<p(ii j2( Y i - E ^)ii < a - vww E( y * - EY ^\ - 0- - £ )* + 2A ) 

i=l i=l 
n 

+ P(||£)(i2 i -EJ2 i )||>rt)- (5) 

8=1 

We would like to apply Corollary [T] to bound first summands on right hand sides 
of the above inequalities and the Markov inequality to bound the deviation of 
the remainder. Thus the aim is now to find a good truncation level p, which 
would ensure that A ~ et and both 

e -t(i-o(*))/jrma* ( ||r i - E r 4 |U )P( ^ || ^(R, - ER t )\\ > et) 

1 - ~ n i=i 

are of order i _p Emaxi< n ||Xi|| p , where K = K (rj, S) is the constant from Corol- 
lary [T] 

Recall now the classical Hoffmann- J0rgensen inequality (e.g. [10] . Chapter 
6, Proposition 6.8., inequality (6.8)), which asserts that 

k 

E max || V < 2 ■ 4 P E max \\Ri\\* + 2(4t ) p , 

Kk<n — ' Ki<n 
~ ~ 1=1 

where 

fe 

to = inf{t > 0: P( max || Vi?^ > t) < (2 ■ 4 f T 1 }. 

Kk<n — ' 
- - i=l 

The idea (following [5]) is to truncate at a level which would allow to take to = 0. 
Let us thus set 

ff = 2-4 p -E max \\X t \\ p . (6) 

l<i<n 

Then P(max 1 < fc <„ || Eti^ll > °) < P(maxi<i<„ \\X.,\\ > p) < (2 • 4 P )- X by 
the definition of Ri and the Markov inequality. Thus 



E max || V R^p < 2 • 4 P • E max \\X, 

l<k<n ^ — 4 Ki<n 



6 



and also 

fc 

E max W^RiW < 2^ • 4 • (E max ||JM p ) 1/p - 

l<fc<n * — * l<i<n 

Notice, that we can assume that Emaxi< t <„ < e p t p , since otherwise 

the statement of the corollary becomes trivial (e is a function of S, so it can be 
used to define C such that right hand sides of the inequalities in question are 
greater than 1, if Emaxi<j<„ > e p t p ). 

Therefore we have 

fc 

A = E max || V R. t \\ < 8te. (7) 

_ _ i=l 

Also, by Jensen's inequality 

fc fc 



E max || - ERi)\\ p < 2 P E max || R t \\ p < 2 • 8 P E max \\Xi\\ p , 

l<k<n — ^ Kk<n ^ — ' Ki<n 

i=l 



so Markov's inequality gives 

P( max || Vfifc - Eifc)|| > rt) < 2 • 8% - P !ggirf_! , 

- - i=l 

Combining the above estimates with inequalities (|4]), ([5]) and ([7]) gives 
fc 



max \\J2Xi\\>(l + v)®Z + t) 



i=l 

k fc 

<P( max || - EY i)\\ > (1 + ~ EYj)|| + (1 - 33e)t) 

- ~ n i=i i=i 
.EmaxKKn \\Xi\\ p 



2 • 8 p e~ 



and 



P(Z < (1 - r?)EZ - i) 

n n 

<P(|| £(y« - EYi)\\ < (1 - ^)E|| - EyO|| - (1 - 17e)t) 



i=l i=l 

:maxi<i<„ \\Xi\\ p 



2 • 8 p e" p - 



t p 



Let us notice that for every / in the unit ball of B* we have E[f(Yi~EYi)] 2 - 
E[f(Yi) - Ef(Yi)} 2 < Ef(Yi) 2 < Ef{Xi) 2 . Thus, by Corollary [1] we can write 
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' max || y^Xi\\ > (l + rj)EZ + t), P(Z < (l-rfiEZ-t) 

' Kk<n — ' 



< exp — 



i 2 (l-33e) 2 



2(1 + 5)a 2 



exp 



(1 - 33e)t 



2K( v ,S)p 



•2-; 



tP 



exp 



Since exp(— x) < C p x p , we have for e < 1/66, 
(1 - 33e)t 



2K( v ,S)p 



<C p [K( Vl S)p]Pt-P = C P K{ V , Sf ■ 2 • 4? - 



■ maxi<i< 



\XA\p 



t" 



To finish the proof it is enough to apply the last two inequalities with 5/2 
instead of 8 and e small enough to ensure that 



(l+V2)(l-33e)- 2 < [1 + 5). 



□ 



As announced, we would also like to state a similar result, obtained in [2], 
valid for variables satisfying stronger, exponential integrability conditions. We 
will use it to obtain concentration inequalities for ||T n || (see Proposition [2] in 
Section 13.31 below) . 

Theorem 3. In the setting of Corollary [IJ assume that for some < a < 1 
and all i, \\Xi\\ip a < oo. For all < r/ < 1 and 8 > 0, there exists a constant 
C = C(a, rj, 8), such that for all t > 0, 



P(Z > (l + r))EZ + t) 
< exp 



2(1 + 5)a 2 



3 exp 



C|| maxj || X 



* II II V« 



and 



HZ < (1 - r))EZ - t) 
< exp 



2(l + 5)a 2 



3 exp 



C\\ max, ||X||||.0 c 



3 Random Toeplitz matrices 

In this part we will focus on the initial problem and prove several results, con- 
cerning the operator norm of random Toeplitz matrices. The consecutive sec- 
tions will be devoted to properties of the expectation, almost sure behaviour 
and concentration inequalities. 
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3.1 Expectation estimates 

We will start with some estimates of the expected operator norm. In the proof of 
our results we use an argument introduced by Meckes, but apply it conditionally. 

Following p~2] , let us first note that T n is a submatrix of the infinite Laurent 
matrix 

L n = [^i-ll{|i-j|<n-l}]i,jeZ; 

so ||T n || < || in ||, where by \\L n \\ we denote the operator norm of L n acting in 
the standard way on ^(Z). If we use the Fourier basis to identify ^(Z) with 
Li2[0, 1], it turns out that L n corresponds to a multiplication operator, with the 
multiplier 

n-l 

f{t)=X + 2j2cos(2njt)Xj 

3 = 1 

and thus ||L„| = ||/||oo- 

Theorem 4. If Xq,Xi, . . . are independent mean zero random variables, then 



EIITJI < EIILJI < C A 



\ i=0 



where C is an absolute constant. 



Proof. By the arguments preceding the formulation of the theorem and the 
standard symmetrization inequality, we have 

E||T n || <E||L n || <2E sup \Y t \, 

i£[0,l] 

where 

n-l 

Y t = X e + 2 ^2 cos(2Trjt)e j X :j 
i=i 

and Ei, £2, ■ ■ ■ is a sequence of i.i.d. Rademacher variables, independent of X/s. 
Using the fact that Rademacher variables are subgaussian and Dudley's entropy 
bound, conditionally on Xj's, one gets 

E E sup \Y t \ < \Y \+C / y/logN([0,l],d,s)ds, (8) 
te[o,i] Jo 

where C is an absolute constant, 



d(h,t 2 ) = \\Y tt - Y t2 \\ 2 = 2 ^ A-?[cos(27rjti) - cos(2^ji 2 )] 2 

and N([0, 1], d, s) is the smallest cardinality of an s-net in [0, 1] with the metric 
d (see e.g. [2], chapter 2.2). 
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We have diam([0, 1], d) < X j =■ D - Moreover 



d(h,t 2 ) < 4vr A 



Y,pxf\h-t 2 \, 

\ 3 = 1 



SO 



N([0,l],d, s)<C 



A 



for s < D, where A = a/EJ=i x ff- 



Therefore 



r°° r u / r>A 

/ y/logN([0,l],d,s)ds< / J log ds 

Jo Jo V s 

t 2 e- t0 -' 2 dt 



< 



CA 

'^2 log (CA/D) 

CA 

V2 



(V21og (CA/D) + V2^j e~ log [CA,D) 



< D^\og (CA/D) + y^D. 



We have A/D < yY^=i J 2 — Cn z / 2 , so the above inequality together with 
((SJ implies 



E e sup |y t | < \Y \+C, 
te[o,i] 



\ 3=1 



Taking into account that Yq = -^o£o + 2 X)J=i ^i e jj integrating with respect to 
the variables Xj's and using Jensen's inequality, we get 



E||L B || < C, 



\ 3=0 



□ 



The following lemma is a partial converse to the above theorem. It also relies 
on arguments from [12] . 

Lemma 1. If Xq, X\ 1 . . . , are i.i.d. and 

hmsup —== < oo, 

n->oo \Jn log n 



then EX 2 < oo. 
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Proof. Without loss of generality we can assume that Xi's are symmetric with 
positive variance. Thus for R large enough Xjl^ Xj \<R} is not identically zero. 

Let ei, £2, ... be i.i.d. Rademacher variables, independent of X^s. By as- 
sumption, for some C < oo, 

E||T n || < Cy/nfog~n~. 
By another argument from [TJ] (see the proof of Theorem 3 therein), we have 



|T n || > sup 
te[o,i] 



X + ^2 2 ( : ~ 0l n ) X 3 cos(2tt jt) 



Thus, using symmetry of Xj's and the contraction principle, we obtain that for 
each R < oo, 

L«/2J 

E||T„|| >E X E £ sup £jXjl {lXjl <R} cos(2n jt) 



te[o,i 



Now, by Proposition 6 of [T2], we obtain for i? large enough 



E||T n || >cE||(0 i )j=l,... > Ln/2j||24/l0g 



ll(aj)i=i,...,L«/2jl|2 



I (ay) 



j-=l,...,|n/2j II 4 



where cij = Xjl{\Xj\<R}- Let A = EXol{|x |<-R}- By the law of large numbers, 
for n large enough (depending on R), the random variable on the right hand 
side is with probability at least 1/2 greater than or equal to 



Thus, for n large enough (depending on R and A), we have 



VnAx/logn 1 / 16 < VnAy login 1 /s^/A) < s/2Cc~ 1 sJn log n, 
which gives A < 32C 2 c~ 2 . Since R is arbitrary, it implies that EX$ < 32C 2 c~ 



□ 



3.2 Almost sure behaviour 

Before we proceed to the analysis of the almost sure behaviour of the operator 
norm of T n , let us introduce some notation that will be used throughout the 
rest of the paper. 

For every positive integer n and i = 0, . . . ,n— 1, let A^ — [l{\j-i\=i}]j,i<n- 
We thus have 

n-1 

T n = Y,Xi4 n) - (9) 

2 = 

This basic relation will allow us to apply to T n the results for sums of indepen- 
dent random variables, presented in Section [3J 
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3.2.1 The non i.i.d. case 

As the first example of application of Theorem [5] we will prove the following 

Proposition 1. If X\. X 2l ■ ■ ■ are independent (not necessarily i.i.d.), EX n = 
and for some a > 1, sup„E|X n | 2 [loglog(|X„| Ve e )] Q < oo. then with probability 
1, 

\\r n \\ n\T n \\ . 

hm sup : — lim sup : < oo 

y/n log n 

Proof. The inequality follows immediately from Theorem[4] We will first prove, 
that 

lir n || gjjiy 

limsup < Limsup a.s. (10) 

n-«x> vnlogn »-»«> Vnlogn 

It turns out that to obtain the inverse of (|10p . we will need a corresponding 
result for Laurent matrices, i.e. 

jjLj Ej|£„jj 

lim sup : < lim sup : a.s. (11) 

n-^oo Vnlogn n->oo V n log n 

The proofs of both statements are very similar. Therefore we will present in 
detail only the argument for T n and then we will indicate all the changes one 
has to make in order to obtain (fTT]) . 

Let us denote A = lim supE||T n ||/-\/n logn. To obtain (fTOj) . it is enough to 
show that for all e > 0, 



lim P( sup||T fc ||/Vfclogfc > A + e) = (12) 

n— oo V k>n / 

We have for 9 > 1, 

Pf sup||T fc ||/v/Hogfc >A + e^j 

V P( max llTdl > {A + e)j9 k \oge k \ 

fc>|k>gn/log0J _ 



^ P(||T flfc+ i|| > (A + e)\jQ k \og 9 k j, (13) 

k> [log n/ log #J 

where in the last inequality we used the fact that ||Tj|| is increasing with i (since 
Ti is a submatrix of Tj+i). Now, for rj small enough and k large enough, we 
have 



(A + e/2)y/6 k \og9 k > (A + e/4)(l + 2r l )y/6 k \og9 k 



>(l + 2n)E||^ fe+1 |L/0- 1 



(fc + 1) log0' 
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which for 9 close to one and k large enough is greater than or equal to (1 
?7)E||Tgfc+i || . Thus, to prove (fT2]l . it is enough to show that 



E 

k=l 



>(\\T eh+ 4 >E\\T ek+1 1|(1 + ^+£^0* log 0*/2) < 



DO 



for arbitrary 9 > 1 and e, 77 > 0. Let us now denote ^(x) = x[loglog(x V M)] a 
and &(x) — x/[loglog(a; V M)] a , where M is a parameter and note that for M 
large enough, '5 is convex and nondecreasing and <I> is nondecreasing. Moreover 
for some constant K (depending on M and a) and all x > 0, 

K + ${^(x)) > x/K. 

We will now take into account the relation ^ to treat T n as a sum of indepen- 
dent variables in the space of matrices and apply Theorem [2j Let us notice that 
since 11^4^ || < 2, a 2 from this theorem is bounded by A9 k+1 sup i<efc+ i EX 2 < 
K\9 k+1 for some constant K\ (depending on the supremum in the assumption 
of the proposition). Therefore, applying Theorem [2] with p — 2, we obtain 



*( 



T ek+ i || > E||T efc+ i || (1 + rj) + ev 7 ^' log 6>72) 

_ h £^ 1 Emax 1<9k+ i \Xi\ 2 
< e R 2 + a 2 



e 2 9 k k log 9 



The series corresponding to the first term on the right hand side is clearly 
convergent. As for the second term, by the integrability assumption on X^s, we 
have for some constants C\ , C2 , 



K^E max |X,| 2 < K + $(*(E max |XJ 2 )) < if + $(E V *(|XJ 2 )) 
< J fC + $(6i fe+1 Ci) < X 



9 fe+1 C 2 



log" fc ' 

which shows that also the other series is convergent, as 

00 ^ 

^klog a k <0 ° 
fe=i 6 



for a > 1. This proves ([12]) and thus also ([TO 



Let us now indicate the changes one has to make in the above argument, to 
prove (llip . First, let us notice that the matrices L n are sums of independent 
variables with values in BiliiTL)) - the space of all bounded operators on the 
Hilbert space £2^). Indeed, we have 

n-i 

L n = J2 x i A i, (14) 

i=0 
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where A, = [l{|<-j|=<}]i,jez< 

The space B{ti(li)) is not separable, but one can still apply Theorem [TJ 
since the norm on this space is expressible as a supremum over a countable set 
of functionals, so one can approximate it by norms in separable spaces. The 
second difference is the lack of monotonicity of \\L n \\ with respect to n, which 
prevents us from using a counterpart of the equality (|13p . This is however not 
a real problem, since, denoting A = limsup ||L n ||/yVi logn, one can write 

p( sup||L fe |l/v/Hogfc > A + e) 

\ k>n ' 

< V P( max \\LA\ > (A + e)y / e k log9 k ) 

fc>Llogn/logeJ _ 

< V P( max \\Li\\ > (A + £) v / 6 k log0 k ) 

fe>[logn/logeJ - 

and use the full strength of Theorem [TJ which gives a bound for the tail of 
maxima of partial sums. The rest of the proof is the same as for T n (note that 
we still have ||Aj|| < 2). 



What remains to be proved is the inverse inequality, i.e. 

\\T n \\ n\T n \\ 

hmsnp > Umsup (15) 

n->oo \/n\ogn n-^oo ^/nlogn 

This can be established again by arguments similar as in [6], Section 4.3 (see 
also pp, Theorem 7). First, let us define X' n = X n l^ Xn<{< ^ n i og „ } and let T' n 
(resp. L' n ) be the Toeplitz (resp. Laurent) matrix generated by X n . We have 

Hm sup ^ = lim sup J$l a.s. (16) 
yn log n ynlogn 

and 

hm sup -^1^=^= = hm sup —JLJlJl- . (17) 

n-«x> Vnlogn oo y'nlogn 

To prove the first equality, it is enough to notice that by the Chebyshev 
inequality 

Y^nxn * x' n ) < ^p(|x„i > v^Toi^) 

n n 

< ^ c E|X ra | 2 [loglog(|X w |Ve e )]° < ^ 
~~ ^— J n[logn][loglogn] a 

which by the Borel-Cantelli Lemma implies that with probability 1, X n — X' n 
for large n. The equality (I16|) follows by ([9|), an analogous formula for T' n and 
the fact that HA^II < 2. 
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Let us now turn to the proof of (fTT|) . Notice that 

n-l 

E||T„||-EK||| <^||A[ Tl) ||E|X l |l {|Xt|>v/n ^ } 



i=0 

n-l m|x ., 2 



<2^E|X 2 |l {|Xi|>v ^ } <X^ 



i=0 VHoil 



= o(Vnlogn), 



which already implies (|17jl . 

To prove (fT5"|) , it is thus sufficient to show that sup„ ||L^||/v / nlogn is inte- 
grable, since then, by Fatou's lemma (recall that ||T^|| < ||£^,||), we get 

IIT'II EIIT'II 
E lim sup > limsup- "" 



y/n log n n \/n log n ' 

which allows us to finish the proof, since by (jTUJ) , (| 16[) and (jTTJ) , we have 

IIT'II EIIT'II 
limsup '' "" < limsup '' a.s. 
n y'nlogn n y'nlogn 

To prove the desired integrability, notice that by Theorem dj (jTTJ) , (flT]) . the 
inequality ||Aj|| < 2 and the fact that with probability 1, X n = X' n for large n, 
we have 

\\L'J 

sup —p=i^= < oo a.s. 

n v n log n 



Moreover Esup n \\A n X' n \\/^/n\ogn < 2. By Proposition 6.12. from [TO], this 
implies that also 

III/ II 

Esup ." "" = < oo, 
n v n log n 

which ends the proof. □ 



We would also like to remark that if one considers higher iterates of log- 
arithm, the assumptions of Proposition [TJ may be weakened, for instance to 
sup n E|X„| 2 [loglog(|X„| VM)][logloglog(|X„| VM)] Q < oo for some a > 1 (the 
proof is essentially the same, the only difference is in the involved series). The 
proposition is however no longer true if we assume only sup„ E | X n | 2 [log log ( | X n \ V 
e)] < oo. To see this, notice that since ||T n || > |X n _i|, by the Borel-Cantelli 
Lemma a necessary condition for the almost sure finiteness of the limsup in 
question is 

£>(|X n | > Cy^logn) < oo 

n 

for some C < oo, which is not true for instance if X n = ± \Jn [log n] [log log log n] 
with probability p n = (nflognjpoglognjpogloglogn])" 1 and with probability 
l-2p n . 
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3.2.2 The i.i.d. case. Proof of main results. 



We will now prove the main results of the article, announced in the Introduction. 
We state them again below as Theorem [5] and Corollary [3] The proofs go along 
the same lines as in the non i.i.d. case. The main tool is Theorem [T] (now we 
can actually use the original formulation from [B] , with sums of p-th moments 
instead of the p-th moment of the maximum of |Xj|'s). 

Theorem 5. Assume that Xq, X\, X2, ■ ■ ■ are i.i.d. random variables. If EXq 
and < EXq < 00, then 

hm — = 1 a.s. 

Proof. Just as the proof of Proposition [TJ the argument is influenced by the 
work of Einmahl and Li. The first step is the truncation. 
Let us define X' n = X n l^ Xn ^ nlogn ^ . We have 



E 1 



\x n ^x' n )=j2 nx > v^ioi^) 



IE 1 {nlogn<^} < KEXl < CO 



so with probability 1, X n = X' n for large n. 



By Theorem 01 we have E||T n || < C \/n log n for some C < 00 (depending on 
the distribution of Xq). Also, by Theorem 3 of [12], symmetrization inequalities 
and Jensen's inequality, we have 

n—1 n—1 

E||T n || >2- 1 || J2 4 n) Xi*i\\ > Z-^IMW E A- n) £ill 
> cE\Xo\y/n logn = cy/nlogn, 

where c is universal and c = 3E|Xq|. 

Let T' n be the Toeplitz matrix corresponding to Xq, . . . ,X' n _i. Since X n = 
X' n for large n and by the above estimates E||T n || — > 00, it is enough to prove 
that with probability 1, 

IIT'II 
lim zt7t= , = 1 . 
n E TJ 
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We also have 



E||T„||-EK| 



< E \\Tn\\ - \K\ 



<n\T n -K\ 



= E||^Ar ) X l l {|Xs|>yn Hp } || 

n-l 

<2E|X | + 2^E|X l |l {|Xs|>yn5p} 



n— 1 ^ 

< 4E|X | + 2E|X | 2 V -=== = o( A /nlogn) = o(E||T n ||), 

(18) 



which shows that E||T„||/E||T^|| — > 1, so it is enough to prove that with proba- 
bility 1, 



\\T II 
lim^^ = 1. 
n E H! 



(19) 



For simplicity let us denote s n = E||T^||. Notice that by the above inequali- 
ties, we have 



zy/nlogn < s n < Cy/n log i 



(20) 



for some < c < C < oo. Fix 9 > 1 (but close to 1) and let m n be the smallest 
integer k such that Sfe > 9 n . We have m„ — > oo. 
Notice that we have 



Sn+l - S n < E, 



^2(X 4 ')2<y , 2nEX 2 =o( S „). 

\ i=0 



Thus 



< 1 + a„, 



where a„ — > 0, so s mn /6 n — > 1. In particular s mn+1 /s TOn — > 0. 

Having established these preliminary facts, we can proceed with the proof of 
(fTO|) . We will show separately that limsup ||T^||/s„ < 1 and liminf ||T^||/s n > 1 
a.s. 

We want to prove that for every e > 0, 



lim p( sup > 1 + e) = 

! ^°° ^ k>n Sk 



and 



I IT' I 



n— >oo V fc>ri s/j 



lim P inf 



<l-e J =0, 
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which will follow if for some 9 > 1 we show that 



k 

and 



Vp( max ^->l + e)<oo 

* — ' V m k <t<m k+1 Si I 



2X 



, m k <i<m k+1 Si 

k 



WW 

< 1 - £ < 00 



Similarly as in the proof of Proposition [T] we can now take advantage of the 
monotonicity of ||X^|| (and s n ), which allows us to replace the above conditions 

by 



$>(ll T k +1 ll >(l + e)s mk ) <oo (21) 

fc 

and 

E P (ll T ^JI <( 1 - £ )^ + i) < TO ( 22 ) 



Let us now notice that since EX; = 0, we have 



n — 1 n— 1 

IMi = H^^x^ix^vTO}!! = II EA-"^ 1 ™ : 

i=0 i=0 
n-1 

< E|| E A^Xil^^^^W = o(E\\T n \\) = o( s „), 



where in the third equality we used (|18p. 
Since s TOfc+1 /s mfc — > 6>, for large fc, 

KJI > 11^ - ET^JI - ss mk+1 /4, 
ll^ +1 ||<||T^ +1 -ET^ +i ||+ eSrofe /4, 

so (j2"Tj) and (|22[) will follow if we show that 



E P (ll T ™. +1 - ET ™* +1 ll > (l + 3e/4) Smfc ) <oo (23) 

and 

Ep(||^-ET4J <(l-3 £ /4> TOfe+1 ) <oo. (24) 



Finally for # ~ 1 (depending on and large we can write 

(l + £/4)< Srofe (l + £ /2), 
E||T^ - ET4J > s m ,(l - e /4) > s m , +1 (l - e/2), 
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so the proof of l|23p and can be reduced to showing that 

5>(||2^ +1 ~ ET UJ\ > (l + Vi)n\TL k+1 -ET; nk+i \\+e Smk /8) < oo 



k 

and 



J2 ] 

k 



< OO, 



»(||T^ - ET^J < (1 - ffc)E||I^ - ET4J - es mk+1 /8) 

k 

where 

l + r/i = (l + 5e/8)(l + e/2)-\ 
l-r? 2 = (l-Se/Sjll-e^)- 1 . 

Now we can use Theorem [5] with p = 3 to obtain 

p(||T r ' fc+i -ET^ +1 || > (l + m )E\\T^ k+i -ET^ k+i \\+8s m j8) 

,2 2 TOfe+iEIXol 3 !, 

— J 

Km k+1 J s^e 3 



< ex p( --e^- 4 -) + x ^r~73 • ( 25 ) 



Similarly 

P(||T^ - ET^J < (1 - r? 2 )||T^ - - £Smfc+1 /s) 

<r v ( tf™*±l_\ _l ^ ^E|^ol 3 l{|Xo|<, 



(26) 



By (|20[) and the definition of s k we now obtain, that for large k, V m fc log rrik ~ 
fe and s mfc ~ fc , so we also have ~ 9 2k /k (where by ~ we mean that the 
quotient of the left and right hand side is bounded away from and infinity by 
some constants, perhaps depending on 9, but independent of k). Thus the right 
hand sides of (f2"5| and (|2"B"|) are bounded by 



K Q2k+2 k -l) 3k f, £ 3 

e 2 fcx e 2 E\x \ 3 i {lXol < cek} 



9 k ke 3 



The series corresponding to the first term on the right hand side is clearly 
convergent. As for the other term, we have 

E g^Wggn < E | Xo |3 E ^<e|x |3.^ 

k ' k-.e k >\x \/c 1 01 

= XC*E|X | 2 < oo, 

which shows (|2"Tj) and (|22p and thus proves the theorem. □ 
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Remark Actually, in the above proof one may as well use the inequality for 
bounded variables (e.g. Corollary [1]) . It is enough to notice that all the other 
calculations will still work if we truncate the variables at the level y/n instead 
of \Jn logn. 



As a corollary we obtain a law of large numbers for the i.i.d. case, proven 
originally in jj. 

Corollary 2. Assume that Xi are i.i.d. random variables, such that EXq < oo 
and EXq = m. Then 

y \\Tn\\ . . 

lim J = \rn\ 

n — >oo fi 

Proof. The argument we present comes from [12]. We repeat it for the sake 
of completeness. Let T' n be the matrix generated by the variables Xi — m, 
i = 0, .. . ,n— 1. By Theorems [4] and [5l we have 



lim sup 



IIT'I 



\jn log 



< oo. 



But T' n = T n — ml n , where I n is the n x n matrix of ones. Since = n, we 
have 



ITJI - \m\n 



< 



IT' I 



which proves the corollary. 



□ 



Remark A similar corollary for Toeplitz matrices generated by independent 
but not identically distributed random variables with the same mean may be de- 
rived from Proposition[TJ Also, if we assume for instance that E|X n | is bounded 
away from 0, we can combine the proofs of Proposition [T] and Theorem[5]to ob- 
tain the law of large numbers (with the normalization E|jT„||) in the centered, 
non i.i.d. case. 



Combining Theorem [5] with Lemma [T] gives the necessary and sufficient con- 
ditions for the limsup ||T„|j / \/n\ogn to be almost surely finite. 

Corollary 3. Assume that Xi are i.i.d. random variables. Then 

\\Tn\\ 

lim sup —== < oo a.s., 

jwoo Vnlogn 

if and only if EXq = and EA"q < oo . 

Proof. The first part follows from Theorems 2] and [5l To prove the second part 
it is enough to show that EXq < oo, since then Corollary [5] gives EXq = 0. To 
prove the finiteness of the second moment, without loss of generality we may 
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assume that the variable is symmetric. The square integrability will follow by 
Lemma [1] if we prove that 

n\T n \\ ^ 

urn sup < oo . 

n— oo Vnlogn 

This can be proven in the same way as part (a) of Lemma 4.1 in [5] (this 
lemma deals with partial sums of i.i.d. variables, whereas in our case T n = 
Y^i=o AjXj, however the fact that ||^ n ' ) || are bounded, uniformly in i and n, 
makes the same argument valid also in our situation). □ 

Problem In view of results for other types of random matrices, a natural 
question arises, whether J^log n * s anTL0S t surely convergent. From Theorem [S] 

EIIT II 

it follows that this is equivalent to the existence of lim , . 

A \Jn log n 

3.3 Some concentration inequalities 

We would like to conclude by presenting some concentration of measure results 
for||T n ||. 

Again, we will look at the matrix T„ as a sum of independent random vari- 
ables in a Banach space. However, since the weak variance a 2 , defined in Section 
[5]is (in the i.i.d. case) up to a universal constant the same as the so called strong 
variance, defined in our situation as 

n— 1 n 

E 2 = ^E||A t || 2 X 2 < 4^EX?, (27) 

i=0 i=l 

we will pass to sums of real random variables. At the cost of increasing some 
constants by a universal factor (in the i.i.d. case), this will allow us to obtain 
concentration around E||T„|j rather than bounds on the probability of deviation 
from (l±77)E||T n ||. 

Our main technical tool will be the following symmetrization lemma, proved 
in [5] (it actually seems to be a part of the folklore, but we haven't been able 
to find any references). 

Lemma 2. Let cp: K — > M be a convex function and G = /(Yi, . . . , Y„), where 
Y\ , . . . , Y n are independent random variables with values in a measurable space 
£. Denote 

Gi = /(Yi, ■ ■ ■ , Yi_i, Yi, Yi+i, . . . , Y n ), 
where (Yi, . . . , Y n ) is an independent copy of (Yi, . . . , Y n ) and assume that 

\G-Gi\ < Fi(Yi,Yi) 

for some functions : £ 2 — > R, i = 1, . . . ,n. Then 

n 

E<p(G - EG) < Eip(^2 SiFi(Yi, %)), 

i=l 
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where e±, . . . ,s n is a sequence of independent Rademacher variables, indepen- 
dent of (Yi)f =1 and (Fj)?=i- 

Thus, whenever we have a concentration inequality for sums of centered real 
variables, which has been obtained by (or can be rephrased as) an estimate for 
expectations of one or many convex functions (like the Lagrange transform or 
moment estimates), by using the above Lemma we can obtain a corresponding 
result for more general variables. In most cases the inequalities for sums of 
Banach space valued variables obtained with this method will be much worse 
than the optimal ones, since they lose the information about the weak variance, 
in our case however the weak variance satisfies 



<j = sup 



E 



E 



U0lh<lfc=O \i-j\=k 



SU P (EE ^ 7fe (EX fc 2 ) 1/2 

a|l2,ll/3|l2,ll72ll2<l u U -l_7. 



sup 

I|7||2<1 



h<-ii(^iwi) 1/! W 



and in the i.i.d. case the choice of 7^ = for all k gives a 2 > ri&X^. 

In the following proposition we present inequalities for Toeplitz matrices 
generated by various classes of independent variables. The proof relies on trans- 
lating proper inequalities for real variables into a moment form (which in this 
case can be easily achieved with integration by parts), using Lemma [2] and then 
going back to tail estimates by optimizing in p the Chebyshev inequality for 
p-th moments. We will skip the details which are quite standard but lengthy. 
The real line inequalities we use are the ip2 inequality for centered variables and 
Theorem [3] 

Proposition 2. Let Xq, . . . , X n _i be independent random variables and let T n 
the corresponding random Toeplitz matrix. Then 

(a) if \\Xi\\^ 2 < 00 for all i, then for all t > 0, 



•(III 



Tn 



\\T n 



K T,i=o\\ x i 



where K is a universal constant, 
(b) if for some a € (0, 1], H-XiH^ < 00 for all i, then 



\T n 



n\T n 



>t) < 2 exp 



1 .ft 2 
mm — 7T . 

K a V£ 2 



t 



max; \X 



where E 2 is defined by formula p?7|) and K a depends only on a. 
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Remark Obviously one can apply the same scheme to other inequalities. We 
would also like to remark that in the non i.i.d. case one may prefer to use 
Theorem [3] for sums of independent Banach space valued variables. At the 
moment we do not know whether also in this case a 2 and S 2 are of the same 
order. 

3.4 Final remarks 

In the article we restrict our attention to symmetric Toeplitz matrices. However, 
the same methods provide analogous results for nonsymmetric Toeplitz matrices 
[Xi_j\i<i j<„ where Xj, i S Z are independent random variables. Therefore, 
the methods apply also to random Hankel matrices [Xj + k-i]i<j ,k<m since (as 
noticed in [5], Remark 1.2.), the singular values of H n are the same as of the 
nonsymmetric Toeplitz matrix, obtained from H n by reversing the order of 
the rows. The results about the expected operator norm of the nonsymmetric 
Toeplitz matrix translate thus directly. As for Theorem one has to be a little 
bit more careful. This result does not translate formally, since the Toeplitz 
matrices corresponding to H n are generated by a permutation of the original 
sequence of random variables. One can however still use all the concentration 
of measure results and simply repeat the proof. 
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